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ABSTRACT 



This study, by means of an experimental design, examines 
whether the use of an electronic manual is more effective and efficient than 
a print manual for the training of novice users with a music score editor 
program. Differences in access to the two types of manuals were looked for, 
and information on users' perceptions regarding the materials they used was 
solicited. A minimalist print manual, a hypermedia manual and two 
questionnaires were developed. The most prominent difference between the two 
manuals was the use of digital video to present procedural information in the 
electronic version. The rest of the information remained the same for the two 
manuals, as did the training strategy based on guided exploration. The 
research findings show an effect of the hypermedia manual in learning 
outcomes, access to manuals and users' perceptions. This effect might have 
been due to the confluence of several factors differentiating the manuals: 
the procedural information in the video format and the design of the 
interface- -with limited control elements, fast access to information, and a 
low complementary interaction between presentation modalities- -minimized the 
potential cognitive overload and allowed the user to start working rapidly. 
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Abstract 

In this study, and by means of an experimental contrast design, we have tried to determine whether the 
use of an electronic manual is more effective and efficient than a print manual for the training of novice 
users with a score editor program. We also looked for differences in access to the two types and for 
information on users’ perceptions regarding the materials they used. Were developed a minimalist print 
manual, an hypermedia manual and two questionnaires. The most prominent difference between the two 
manuals was the use of digital video to present procedural information in the electronic version. The rest 
of the information remained the same for the two manuals, as did the training strategy based on guided 
exploration. The research findings show an effect of the hypermedia manual in learning outcomes, access 
to manuals and users' perceptions. This effect might have been due to the confluence of several factors 
differentiating the manuals; the procedural information in the video format and the design of the interface - 
with limited control elements, fast access to information and a low complementary interaction between 
presentation modalities- minimized the potential cognitive overload and allowed the user to start working 
rapidly. 

Keywords: novice user training, training materials, hypermedia, minimalism, music technology. 



1. Introduction 

Research related to technical documentation, understood as an instructional resource, investigates design and 
elaboration strategies that are especially important because of their impact on learning. Educational research 
is deficient on empirical studies comparing novice users’ learning outcomes in working with different types of 
documentation. This deficiency extends to a lack of contrasting on presentation modalities. It is widely 
expected that computers will increasingly play an important role as normal working tools in different 
knowledge domains, musical disciplines included (Tejada, 1998). At least in academic environments, a 
considerable amount of time will certainly be devoted to learning computer programs and controlling 
machines. Educators will need to expand their knowledge of methodologies and materials to include this type 
of learning. 



2. Literature and hypothesis 

2.1 Minimalist training materials 



Carroll et al. (1987) observed the novice users' behavior and they found problems on the program 
documentation: it was extensive and exhaustive and did not have either a task-based approach or a system for 
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error detection and correction. In their empirical studies, the model was a learner focused to act, interested 
principally in doing real tasks, trying to make sense of all, he/she sees, hears, reads and does; someone that 
uses their own hypothesis even when the evidence contradicts them; a learner with a tendency to access 
technical documentation in a non-linear way (Carroll, 1990). Conventional manuals do not address this novice 
user's needs. This explains why only a small percentage of users read technical documentation (Lazonder & 
van der Meij, 1993). Novice users, focused on their goals, generally explore program functions by means of a 
trial -and- error strategy, which technical writers try to avoid in the manuals. Furthermore, while users explore, 
they make many errors, which explain the high amount of time devoted to correct them -about 25% to 50% of 
training time (Lazonder & van der Meij, 1994, 1995). The minimalism, a model of training heavily user 
centered, takes on two key principles from cognitive psychology: active learning and constructivism. Users 
can learn more and better if they are actively involved; when they are doing something. Users do not learn so 
much if they follow instructions like a script. Secondly, the user builds up their own mental models 
combining their past experiences with new information; hence, their mental models, already constructed, are 
modified. It makes explicit the user's goals and involves them in real tasks. It reduces the need for extended 
training materials and supports error recognition and correction. Its aims are to maintain motivation, promote 
active learning and make the learning environment user-friendly, that is, it enables exploration that users can 
do without feeling frustrated by their errors. Several authors define principles for the minimalist model 
(Carroll 1990, 1998; Van der Meij & Carroll, 1995).' These share some characteristics with cognitive 
psychology: focus on real tasks; active learning; documentation structure based on users' real needs; brevity 
of materials; error recognition and correction; modularity; screen-documentation coordination system; 
iterative documentation design. Empirical research contrasting minimal materials and conventional ones has 
shown the superiority of the minimalist approach in developing training materials. 1 



2.2 Problems of non-linear information access 

Interacting with information in a hypermedia structure is a complex cognitive activity. Total freedom in the 
choice of the pathway into a hypermedia system may mean that the user's orientation abilities decrease (Dede, 
1992) and a cognitive overload can arise. The term cognitive overload refers to the additional cognitive effort 
and concentration needed to simultaneously perform certain tasks. This demand for additional effort is due not 
only to the user taking decisions about what path to follow, but also to being forced to remember his/her 
situation in the net and the nodes that he/she has visited. Sciarone and Meijer (1993) have shown that the 
freedom provided in navigating through a hypertext system provoked users' inability to do the tasks required of 
them and that they preferred finding the responses through the system’s help rather than learning the system 
itself Several authors have tried to overcome this problem by providing navigational help, such as maps, 
indexes or pathway restrictions (Gay, Trumbull, & Mazur, 1991). Some useful techniques in facilitating 
navigation and avoiding disorientation are guided tours, maps, backtracks, bookmarks, overview diagrams, 
search engines and fisheye views (see Nielsen, 1995). 



2.3 Contrast of print and electronic materials 

Empirical contrast studies of educative print and electronic materials have shown contradictory results, due to 
the heterogeneity of research goals, subjects and type of materials used. Some studies show positive results for 
print materials. 2 There are two studies on comparing still and dynamic images in hypermedia materials. The 
video information included in both was declarative. Christel (1994) compared two courseware versions. A 
software engineering course included several records of meetings and other methodological aspects in which 
students could interact with simulated participants. One version included video at 30 fps and the other version 
presented the same contents (V) with slides which were replaced every 4 seconds (S). In test sessions, group V 



1 A review of minimalism research can be found in McCreary & Carroll (1998). Is essential the work of Carroll 
et al. (1987) and his replication in Lazonder & Van der Meij (1993). For concrete aspects of minimalism, see 
Nowaczyk & James (1993), Lazonder (1994), Wiedenbeck, Zila & McConell (1995) and Van der Grijspaarde 
(1995). A multiple perspective on minimalism can be found in Carroll (1998). 

2 See Horton (1994), Brockmann (1990), Nielsen (1995) and Van der Geest (1994) for a review of contrast 
studies. 
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recalled 89% of the required information while group S recalled 71%. Pane (1994) assessed a multimedia- 
learning environment in the biology domain. The study measured the summative effect on students' 
performance and their satisfaction with the material. Instructional material containing video and simulations 
(V) was compared with other material containing static graphics (G); the results showed that group V spent 
more time working through it and scored better than group G. The author speculates that the time difference 
was probably due to group V projecting the video and simulations more often. These studies suggest that 
video information can enhance learning in a computer-based environment. 



2.4 Hypothesis 

Expressed in negative form, the hypothesis are (dependent variables between parenthesis): 

Hypothesis 1 (HI): When carrying out tasks, there are no significant differences between users of the 
hypermedia manual and users of the print manual (voluntary exercises -time and number- near transfer tasks - 
time and accuracy-, and far transfer tasks-time and accuracy-). 

Hypothesis 2 (H2): There are no significant differences in manual access between users of the hypermedia 
manual and users of the print manual (frequency of access, number of jumps, access time). 

Hypothesis 3 (H3): There are no significant differences of opinion about the materials between users of 
hypermedia manual and users of print manual (ease of use, usefulness, satisfaction, structure and organization, 
speed of information search and the preference for the other manual). 



3 Methodology 

3.1 Previous work 

We gathered data from users over two academic years through verbal protocols, which showed the most 
frequent problems users had with the music score editor, the "impasses" or errors they made, methodologies 
they used to solve problems and whether they needed external help to resolve the impasses. These data 
allowed us to make a task analysis, task selection and an error-recognizing and correction system to include in 
the manuals. 

Our model of user was one with the following profile: low skills in computer use, a moderate level of 
motivation in learning with computers, high motivation in learning music programs and a low-moderate level 
of previous music knowledge. On the basis of this profile and the type of learning intended, we adopted the 
features of the minimalist model. 



3.2 Materials 

Before building the two manuals, we translated the music score editor into Spanish in order to: a) make the 
user's work environment more accessible; and b) eliminate a potential nuisance variable during the 
experiment. 

A task analysis was carried out to determine the tasks to include. In the print manual, the procedural 
information was presented in text, while in the hypermedia manual it was presented in a digital video format. 
The hypermedia manual did not display video with other complementary information (textual or graphic) in 
order to avoid both dividing attention and the appearance of cognitive overload (Sweller, 1994). The manuals 
consisted mainly of procedural information, except for the first unit, which was devoted to the conceptual, 
specific terminology used. This unit was a still graphic without text. The structure of both manual contents 
was logical, sequential and chronological with respect to the task. Contents were split up into twenty units; 
the unit titles made up the Table of Contents and were used as organizers. The interface was quite 
straightforward, with a reduced number of control elements that allowed users to run a complete navigation. 
The hypermedia manual included a help system in order to facilitate operations and minimize the potential 
cognitive overload Ebersole, 1997). 
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According to minimalist model, a section called "Posibles Errores" (Possible Errors) for diagnosing and 
correcting errors was included in eleven units of both manuals, which were most susceptible to syntactic and 
semantic errors (Carroll, 1990; Lazonder & Van der Meij, 1995). The system consisted of a description of 
most frequent error and its general or specific corrections. Also, other section termed "Por ti mismo” (On 
Your Own) was included in order to stimulate user guided exploration and production. With this section, we 
expected users to do voluntary tasks (see Carroll et al., 1987; Van der Meij, 1993). 

Usability tests were carried out with novice users that allowed us to gather data for redesigning the manuals. A 
pilot test was carried out in order to: a) assess the time necessary to accomplish near and far transfer tasks in 
experiment; and b) modify the observers’ score card. The final version of the print manual was a fifteen-page 
booklet, whereas the original manual was a 256-page book. The final version of the hypermedia manual was 
an auto executable (4 Mb.) made using Macromedia’s Director with links to 6.4 Mb. of video clips. 



3.2 Subjects 

To test the fifteen sub hypotheses, we used inter-subject contrast design with volunteer undergraduates on the 
Music Education Teacher course at the University of La Rioja -Spain. An initial questionnaire about previous 
experience of use of computers filtered out thirty subjects (1 1 males and 19 females) with an age range 
between 17-26 years. This sample was regrouped in sub samples corresponding to nuisance variables (age, 
course and sex) and assigned randomly to each of the experimental conditions to balance the groups. The 
subjects were provided with a consent form for participation and received one extra credit. 



3.3 Control of variables 

The documentation to train with a music score editor program (Encore) was the independent variable; it had 
two levels: hypermedia manual (HM) and print manual (PM). To balance the experimental conditions, the 
subjects were grouped in sex, age and course sub samples. They were then randomly assigned to each 
experimental condition. The program that subjects were to learn was translated into Spanish. The researcher 
was neither an observer nor an evaluator. The experimental environment was constant: place, hour and 
computers. 

Each of the fifteen observers recorded measurements on two subjects, one of each experimental condition: 
study time for each instructional unit, time of voluntary exercises and time of transfer tasks in the test stage. 
Observers also recorded the current unit number and problems expressed by subjects ’’thinking aloud”. 
Variables related to the users’ perceptions were measured with a questionnaire working on a seven-point scale. 
An external evaluator scored the test files by an evaluation form provided by the experimenter, which 
indicated scores based on the absence/presence of items required for transfer tasks. 



3.4 Experimental environment and procedures 

The experiment was carried out in a computer classroom at the University of La Rioja with fifteen Macintosh 
PowerMac 6230; it took three training sessions of ninety minutes and one test session (one week between 
each). The print manual was a booklet with fifteen pages of text and graphics. 

All the subjects in each experimental condition were told that the research aim was to study how a user learns 
a music score editor program. They were not informed about the other manual. Based on the data obtained in 
the pilot test, subjects were assigned thirty minutes to do nine near transfer tasks. These included fifteen 
different operations to be undertaken out of context. Subjects were assigned fifteen minutes to do one far 
transfer task. This consisted of replicating a music score and involved the application of knowledge to novel 
situations and the application of non-explicit information from the manuals. Subjects were not allowed to 
consult their manual while performing the transfer tasks (test session). When finished with the test session, 
subjects were provided with a final questionnaire, model A or B depending on the experimental condition, to 
test the dependent variables on users’ perceptions. 



4. Results 
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4.1 Training and learning 



Three PM subjects and four HM subjects did not finish all the near transfer tasks. This difference was not 
significant (means: HM=8,66; PM=8,46). 

Analysis of the data related to learning outcomes with the U-Mann-Whitney test showed significant variance 
(Table 1). AT test for non-related samples was run, which confirmed the significance. According to these 
results, the HM group scored better in near and far transfer tasks than the PM group. 





Tabic 1 

Results of variables related to learning outcomes and their statistical significance 
(standard deviation between parenthesis. HM=hypermedia manual. PM= print manual) 






number of 


mean time for 


mean time for 


mean time for far 


mean score of 


mean score of 




voluntary 


voluntary 


near transfer 


transfer task 


near transfer 


far transfer 




exercises 


exercises 
(In sec) 


tasks 
(in see) 


(in sec.) 


tasks 

(max. 21) 


tasks 
(max. 20) 




18,46 


246,59 


1477,46 


878,86 


19,96 


19,01 


HM 


(6.50) 


(92,89) 


(370,09) 


(57,13) 


0*74) 


(0,77) 




21,06 


245,58 


1431,60 


879,60 


16,90 


13,83 


PM 


(7,45) 


(95.26) 


(390,0) 


(78.96) 


(3,47) 


(2,20) 


Significance 


U=100; 

p>0,05 


U=90; 

p>0,05 


U=106; 

p>0,05 


U=I01; 

p>0,05 


11=43,5; 

p<0,05 

t- 3,05; df=28; 
p<0,05 


U=4; p<0,001 
=8,60; df=17,43 

p<0,001 



4.2 Access to documentation 

This variable was operationalized as a) number of accesses; b) jumps between non-consecutive units; and c) 
time of access. The results for number and time of accesses showed statistical significance with the U-Mann- 
Whitney test (Table 2). A T test for non-related samples confirmed the significance. Group HM consulted the 
manual less and consumed 33% less time than their counterpart. 



Table 2 

Results of variables related to access to documentation and their statistical significance (standard 
deviation between parenthesis) 




Mean number of 


Mean number of 


Total time consulting 




accesses 


jumps between units 


documentation 








(in sec.) 


HM 


49,06 


3,53 


3213,20 




(38,74) 




(1465,63) 


PM 


282,46 


2,60 


4751,53 




(170,81) 




(2113,93) 


Significance 


U=6; p<0,0001 


U=99; p>0,05 


U=65; p<0,05 




t= -5,16; df=15,44 




t= -2,32; df=28 p<0,05 




p<0,001 







4.3 Users’ perceptions 

The item 13 of the final questionnaire asked subjects to evaluate their preferences for the prominent 
information modality included in the other manual, the one they don’t used (Table 3). The users of print 
manual would have preferred to use the hypermedia manual to a greater extent than the HM users the print 
manual. 
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Table 3 

Results of variables related to users’ perceptions and their significance (max. 7) 
(HM=hypermcdia manual. PM= print manual) 




Ease of use 


Usefulness 


Satisfaction 


Organization 
and structure 


speed of 
searching 
information 


Preferences for 
the other 
manual 


HM 


3,06 


6,53 


5,86 


6,00 


2,53 


2,00 


PM 


2,40 


6,33 


5,86 


5,60 


2,33 


3,93 


Significance 


U=93; p>0,05 


U=98; p>0,05 


U=1 12; p>0,05 


U=99; p>0,05 


U=100; p>0,05 


U=47; p<0,005 

t= -3,37; 
df=20,7; p<0,01 



5. Conclusions 

The research findings show an effect of the hypermedia manual in learning outcomes; access to manuals and 
users' perceptions. This effect might have been due to the confluence of several factors that differentiate the 
manuals. First, the procedural information in the hypermedia manual was exclusively video. The hypermedia 
manual approached a modeling of the student using a presentation modality that fitted better with the type of 
learning sought; it produced extra time during training and a better assimilation of task procedures by users. 
Secondly, the design of the interface -with limited control elements, fast access to information and a low 
complementary interaction between presentation modalities- minimized the potential cognitive overload and 
allowed the user to start working rapidly. Designers of educational software materials for novice users should 
cover users' information needs -focus on user- rather than follow a systemic approach, for example by making 
a list of what the software is capable of doing. With the incorporation of novice users to learning situations, 
electronic training materials should be conceived so that the user can quickly build a suitable mental model of 
the software by explicit wordings from domain knowledge, program features and capabilities, through a real 
task approach and the presentation of procedural information in the video format. 

Further investigations could offer users the freedom to access both types of documentation in order to study 
the relationships between use frequency and the variables measured. This will help in understanding the 
reasons why a particular user (expert, intermediate, novice) chooses one or another type of documentation and 
therefore knowing more on the user's information search strategies. Two variables not studied in this work are 
the type and number of errors during training. These ones might be important to explain the effect of 
materials. Though this would require highly trained observers, these two variables and error correction time 
should be studied to relate them to access to materials and learning outcomes. 

This study made a moderate attempt to measure the effectiveness and efficiency of two types of auto 
instructive learning software materials through a guided exploration by novice users. It left at the margins 
important matters like learning style influences, degree of learning directivity, training strategies or type of 
users, which deserve to be investigated. 
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